The present invention relates generally to a video enhancement method, apparatus and computer program, and in particular to a method, apparatus and computer program useful for enhancing the visual quality of videos.
The acquisition process of a video stream often introduces distortions and noise. Video camera introduce electronic noise and blur due to imperfect optics. Other videos such as medical X rays or infra-red videos have other types of noise and the resolution is limited by the acquisition process. In addition, distortions may also be introduced by digital video compression. For example, MPEG 2 or MPEG 4 compression standards introduce block effects and mosquito noise that reduce the video quality. Transport of video over analog channels also incorporates noise into the video signal.
Video noise and compression artifacts can be attenuated with a linear filtering in space and time but this process introduces blur along sharp transitions and fast moving objects. The enhancement process of a video must introduce a limited time delay in most applications. In television, medical and military applications, it may even be necessary to use causal procedures that only process past images in order to restore a video with nearly no delay. Recursive time filters are generally used for this purpose.
To reduce the blur introduced by linear filters, adaptive filtering techniques have been introduced. A parameter adjustment of the time recursive filters is incorporated in order to reduce the averaging when the scene is moving. This parameter adjustment can be incorporated in the more general framework of a Kalman filtering. However, there is no sufficiently reliable model of video images that allows to find robust parameter adjustment procedures. As a result, the range of adaptivity is often small in order to avoid making important errors. Moreover, the parameter adjustment does not take into account the joint time and space image properties.
For an image, efficient adaptive noise removal algorithms are implemented with thresholding strategies applied to the output of a subband transform such as a wavelet transform, or a wavelet packet transform or a bandlet transform. Thresholding subband images is equivalent to adaptively average the input image where there is no sharp transition. Blur removal can also be implemented with a sharpening which increases the amplitude of high frequency subband images, with parameters that depend upon the blur.
For videos, a spatio-temporal subband transform, with a combination of a spatial wavelet transform and a time wavelet transform, replaces the subband transform used for images. Non-linear operators such as thresholding operators are applied to the resulting spatio-temporal subband images and an enhanced video image is reconstructed by combining an inverse time wavelet transform and an inverse spatial subband transform. Such algorithms adaptively remove the noise depending upon the local sharpness and motion of video structures. However, state of the art video processing methods use a combination of a time wavelet transform and an inverse time wavelet transform that introduces a time delay that is typically equal to the maximum time support of multiscale wavelets. To take advantage of time redundancy, this maximum time support must be sufficiently large but this produces a large time delay. The resulting delay is often too large for real-time video enhancement applications, in particular when delays close to zero are required.
Accordingly, there exists a need in the art for improving spatio-temporal subband trans-form methods for video enhancement, by introducing a combination of a time wavelet trans-form and an inverse time wavelet transform that produces a delay d that does not depend upon the maximum time support of multiscale wavelets, and which can potentially be set to zero for causal video enhancement methods.
In addition, many video sources (and in particular medical X-ray video images or defense and security night-vision video images) have a dynamic range that cannot be displayed on the available displays, and applying a sharpening process is useful for increasing the legibility of the video or for making it look nicer. This sharpening process can be applied on a wavelet transform of a video sequence to enhance its local contrast, and there equally exists a need in the art for improving spatio-temporal transform methods for video sharpening or for a combined video enhancement and sharpening with a limited delay d.